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EVALUATION OF PROJECT ALERT 
(ATLANTANS LEARNING EMPLOYMENT RESPONSIBILITIES TOGETHER) 
SUMMER YOUTH EMPLOYMENT PROGRAM 

1991 



Project ALERT was conducted by the Atlanta Public School System (APS) for the 
ninth consecutive year during the summer of 1991. Project ALERT is an education 
component of the Summer Youth Employment Program (SYEP) and represents a 
concentrated effort to address problems associated with the High unemployment 
rate among economically disadvantaged youth. Although the rate of 
unemployment has declined nationally, the rate of unemployment for minority 
youth continues to be greater than that of other groups. 



PROJECT DESIGN 

Project Description 

This project was developed and initiated by APS in conjunction with the Atlanta 
Private Industry Council, Incorporated (PIC). Funded under the Job Training 
Partnership Act (JTPA), Project ALERT was designed to provide service to heighten 
the employability level of 100 students rising to grades 8-12 (14-18 years old). 
Participants were involved in Project ALERT from June 12 through Augusts, 1991. 

The participants in this project were economically disadvantaged applicants 
whose scores on the Georgia Criterion-Referenced Tests (GCRT), BST Preparation 
Tests , or Georgia Basic Skills Tests (GBST) in reading and mathematics indicated that 
the7 were academically at risk. Each participant was selected and referred by the PIC 
Summer Intake staff. All participants were residents of the City of Atlanta, and all 
but ore were enrolled in APS during the 1990-91 academic year. Prior to enrollment 
in Project ALERT, each participant was required to indicate a willingness to 
participate. 

Participants were assigned to various public and private nonprofit organizations 
four hours per day, five days a week for a total of twenty (20) hou. per week. The 
20 hours of work experience were designed to provide meaningful, well-supervised 
opportunities for expansion and/or development of employability skills. 

In addition to the 20 hours of work, each participant was enrolled in educational 
activities for 17i/2 hours per week. These educational activities focused on the 
improvement of reading and mathematics skills. Those participants successfully 
completing the coursework received 1 5 hours of academic credit. In addition to the 
academic classes, career education enrichment activities were provided through field 
trips, assembly programs, counseling. Participants were paid at the rate of $4.25 per 
hour for the 20 hours of work experience. 

Project ALERT was based at the centrally located Kennedy Middle School for the 
instructional portion of the program. The specific worksites were located 
throughout the City of Atlanta at approved agencies. 



Program Objectiv es 

The primary goal of this project was to increase the potential for future 
employment of participants by helping them improve academic performance and 
gain meaningful employment experience. 

Specific goals and objectives were as follows: 

Goal #1 : To enhance the basic education skills of eligible youth (ages 14-18). 

Objective 1 : To provide remedial instruction in reading and 
mathematics to 100 rising ninth through twelfth graders. 
Upon completion, 75 percent of the participants will 
demonstrate an increase in academic performance as 
measured by the ALERT pre- and post-tests. 

Objective 2 : To ensure that improvement in academic performance is 
documented. 

Goal #2: To provide personal management skills training and enrichment 
activities in career education to participants. 

Goal #3: To provide participants with meaningful, well-supervised work 
experiences. 

Performance Standards 

The following performance standards were established for Project ALERT: 

1. All funded slots for participants will be filled by June 17, 1991, on the 
condition that an acceptable number of PIC referrals are made. 

2. Slot vacancies must be filled within three working days of notification of 
termination. 

3. A minimum of 80 percent of the participants will attend at least 85 percent of 
the time. 

4. A minimum of 75 percent of the participants who enroll will complete the 
program objectives (i.e., will receive 15 hours of academic credit). 

5. Dropouts will not exceed 10 percent of total terminations. 

6. A minimum of 75 percent of the program participants will demonstrate 
improved academic performance in reading and mathematics skills. The 
criterion-referenced pretests and posttests developed by the Atlanta Public 
Schools (APS) Department of Research and Evaluation will measure changes in 
competency levels. 

Project ALERT officially started on June 10, 1991. The first few days of the 
program were used for staff orientation and preplanning. Curriculum areas of 
concentration were established, and operational procedures were reviewed with 
staff members. In addition, staff members received test data for use in the 
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appropriate placement of participants for class grouping. Each student's placement 
was based in part upon his/her test scores used to determine eligibility for 
participation in the project. 

Students enrolled in Project ALERT attended three half-day (4-hour) orientation 
sessions, June 12-14. The first session was conducted by APS staff; the second, by PIC 
staff; and the third, by Project ALERT staff. Participants were paid for their 
attendance during the twelve hours of orientation. 

Following the orientation and assignment to homerooms, students were cycled 
through their class schedules. Instructional activities were designed to address 
individual student academic weaknesses, although the overall objectives for all 
students were the same. 

Beginning at 7:45 a. m. each school day, Project ALERT participants were involved 
in academic tasks, classes for enhancing self-esteem, career-oriented activities and/or 
field trips, with one-half hour for lunch. Attending the morning classes was a 
prerequisite for working each afternoon. 



FINDINGS 



Data Collection and Analysis 

The data gathered on Project ALERT participants were analyzed in order to 
determine the degree to which the performance standards were met. The 
coordinator of the project provided enrollment information and copies of grade 
sheets showing students' names and ID numbers, numerical and letter course grades, 
conduct grades, and the number of days absent. Evaluation of performance on 
standards 1-5 was based on the analysis of these data. 

Project participants were administered the Project ALERT pretest during the 
initial days of the program and the Project ALERT posttest one week prior to the 
program's end. These tests were designed as parallel forms of an instrument 
developed to measure performance in reading and mathematics in areas of 
weakness identified from previous GBST results of APS students. Differences 
between participants' Project ALERT pretest and posttest scores were examined by 
comparing the scores obtained on the reading and mathematics subtests and the 
total test scores. Evaluation of performance standard 6 was based on this analysis. 

Although not required by the contract, students' evaluations of the project were 
also obtained. Students present on the last day of the project completed a 
questionnaire on which they indicated how much they liked or disliked eleven 
different aspects of the program, their interest in participating in the program 
again, and suggestions for improving the project. The results of the students' 
evaluations are presented after the results related to the performance standards. 

Results 

The six performance standards are listed below along with the results of the data 
analyses. 
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1. All funded slots for participants will be filled by June 17, 1991, on the 
condition that an acceptable number of PIC referrals are made. 

One hundred slots were funded for Project ALERT in 1991. PIC referred 78 
eligible students, 65 of whom enrolled in the program by June 17. 1991, and 3 
of whom enrolled late. Ten of the referred students declined to enroll. 
Because too few referrals were made by PIC and were not made early enough 
for effective recruitment, this performance standard could not be met. 

2. Slot vacancies must be filled within three working days of notification of 
termination. 

The seven terminations from the program occurred after June 26, the last 
date on which new participants could enroll. Therefore, no slot vacancies 
could be filled, and this performance standard could not be met. 

3. A minimum of 80 percent of the participants will attend at least 85 percent of 
the time. 

An 85% attendance rate was equivalent to attendance on at least 34 of the 
40 days of the project (excluding orientation). Of the 61 participants who 
completed the program, 49 (80%) attended at least 34 days, or 85% of the 
time. This performance standard was met. 

It should be noted, however, that 14 (21%) of the students had perfect 
attendance. 

4. A minimum of 75 percent of the participants who enroll will complete the 
program objectives (i.e., will receive 15 hours of academic credit). 

Of the 68 students enrolled in the program, 56 (82%) received 1 5 hours of 
academic credit (Table 1). This performance standard was exceeded. 



TABLE 1 

ACADEMIC CREDIT RECEIVED BY PROJECT 
ALERT ENROLLEES, 1991 



Credit 
Received 


N 


% 


15 hours 


56 


82 


7.5 hours 


1 


2 


None, but completed the 
program 


3 


4 


None, but did not 
complete the program 


7 


10 


Ineligible (middle school 
student) 


1 


1 


TOTAL 


68 
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5. Dropouts will not exceed 1 0 percent of total terminations. 

Seven program participants, or 10% of the total number of participants, 
dropped out of or were terminated from the program because of 
nonattendance. Four of these participants felt unabie to continue because of 
pregnancy or illness. The reasons for the other three students' absences were 
undetermined. 

Without a definition of dropout for this project, it is unclear whether any or 
all of these terminations should be considered dropouts. Therefore, performance 
on this standard cannot be adequately assessed. 

6. A minimum of 75 percent of the program participants will demonstate improved 
academic performance in reading and mathematics skills. The criterion- 
referenced pretests and posttests developed by the Atlanta Public Schools (APS) 
Department of Research and Evaluation will measure changes in competency 
levels. 

As shown in Table 2, 34 (62%) of the 55 students who took both the pretest 
and the posttest scored higher on the reading portion of the posttest; 51 (93%) 
scored higher on the mathematics portion; and 53 (96%) scored higher on the 
total test. This standard was exceeded in mathematics and overall, but not met in 
reading. 



TABLE 2 

SUMMARY OF CHANGES FROM PRETEST TO POSTTEST SCORES 

PROJECT ALERT, 1991 



Direction of 
Score Change 


Reading 


Mathematics 


Total Test 


N 


% 


N 


% 


N 


% 


Increase 


34 


62 


51 


93 


53 


96 


Decrease 


10 


18 


2 


4 


2 


4 


No change 


11 


20 


2 


4 


0 


0 


TOTAL 


55 


100 


55 


101 


55 


100 



Note : Percentages rounded to nearest integer. 



The results of the student's evaluations are provided in Table 3. Forty-two (69%) 
of the 61 participants who completed the program responded to the questionnaire 
A large majority, 88% or more of the respondents, indicated that they liked "a lo*" 
or "a little each of the eleven aspects of the program included in the questionnaire 
All of them responded that they liked the self-esteem class. 
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TABLE 3 

RESPONSES TO STUDENT EVALUATION (N - 42) 



PROJECT ALERT, 1991 



Please circle the letter that best describes your feelings about each of the following: 

Percent Responding 







Liked 


Lilted a 
Utile 


Uislilced " 
a Little 


Disliked 

a Lor 


Not Sure 


Orientation 


1 


50 


45 


— 


— 


— 


Math class 


2 


55 


33 


5 


5 


— 


Reading Class 


3 


74 


19 


7 




— 


\~ornpuicr iw 
(math) 


A 
*♦ 


60 


Jl 


5 


5 




Computer lab 
(reading) 


5 


67 


24 


7 


2 




Self •esteem class 


6 


oo 










Working 


7 


71 


21 


7 






Being with the 
other students 


8 


67 


29 




2 


i 


Being with the 
adults 


9 


69 


26 




2 


2 


Field trips 


10 


64 


31 


2 


2 




Assembly programs 


11 


60 


38 






2 



2 . Would you want to participate in this program again? 
Yes No ioz Maybe 29* . 
Why? 



3 . What could be done to make Project ALERT better? 



BT. ap 
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When asked if they would like to participate in the program again, a measure of 
overall satisfaction with the program, most (62%) of the respondents answered 
"yes." Of the 21 students answering why they would like to participate again, about 
a third indicated that it was a good experience or an opportunity to learn more. 
Another third responded that the project was fun or simply that they liked it. The 
remaining students gave various reasons, including "only thing to do in the 
summer" and opportunity to "meet new faces." 

Students who were not sure whether they would want to participate again 
generally gave positive reasons for wanting to do so (good teachers, meet a lot of 
people, good time), but one objected to the project starting so early in the morning. 

Only two of the four students who would not want to participate again gave 
reasons for not wanting to do so ("to get more money for school" and not liking it). 

Twenty-four students responded with specific - and sometimes contradictory -- 
suggestions for improving Project ALERT. Most of the suggestions related to 
activities (more trips, more sports, more activities like Project Survival, more work, 
less work), daily schedule (start later in the day, get off work earlier, work more 
hours), and money (more money, more tokens). A few suggested changes in quality 
(better trips, students and teachers work harder, pay attention in class, stop fighting, 
hot lunches), and one suggested including a good breakfast as part of the program. 
The generalized responses included both nothing" and "everything." 



SUMMARY AND CONCLUSIONS 



During the summer of 1991, Project ALERT provided a variety of learning 
opportunities for 68 scudents rising to grades 8-12 in Atlanta. Most (90%) of these 
students completed the program; the mc;ority of those who did not complete the 
program left because of illness. The participants evaluated the program positively, 
and their attendance rate was satisfactory. 

Based upon pretest-posttest scores, students improved both reading and 
mathematics skills, with the greatest improvement demonstrated in mathematics. A 
substantia! majority of the students enrolled in the project also earned 15 hours of 
academic credit. 

Six performance standards were established for the project. Two could not be 
met because of an insufficient number of referrals and the timing of the 
terminations. Three of the performance standards were met or exceeded. One of 
the performance standards was unclear. 

While the level of participation and the acquisition of academic skills were major 
components of Project ALERT, participants also had opportunities to gain valuable 
work experiences. In addition to on-the-job skill development, students learned the 
importance of punctuality, following directions, and cooperation in work settings. 

Based upon these results. Project ALERT met its goals and was deemed a success. 
However, the achievement of performance standards might be enhanceH by the 
following actions: 

12 
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1. Begin the recruiting process eailier, especially in regard to referrals. 



2. Clarify the performance standard relating to terminations and dropouts. 

Furthermore, the benefits derived from smaller classes, more attention from 
teachers and counselors, and structured work experiences combined with academic 
instruction suggest the possibility of integrating Project ALERT with the remedial 
program during the regular school term as a way of maintaining or increasing the 
gains made during the summer. 

R& E/BT;lp/#668 6/9-27-91 
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